Skip to content

Skills read plans and budgets correctly on every surface, and link back to OpenApps - #4

Merged
ataniz merged 1 commit into
mainfrom
improvements-from-verification
Nov 5, 2026
Merged

ataniz merged 1 commit into
mainfrom
improvements-from-verification

Conversation

@ataniz

@ataniz ataniz commented Nov 5, 2026

Copy link
Copy Markdown
Contributor

This round comes from running every skill journey on production data, signed out and on Free and Pro, under Claude and under Codex gpt-6.1-sol.

What changed

  • The references state each plan's limits, Pro's one-call reads and the budget line.
  • A denied npx moves on to MCP or curl before the skill asks the user anything.
  • Codex keeps .agents/ read-only, so the context file falls back to a root plugin-stack.md.
  • submit-plugin and audit-discoverability run on the repo when someone asks how to do the thing.
  • Skill descriptions cover requests that used to miss them.
  • scripts/check-contract.mjs fails when a reference drifts from the live API, on each push and weekly.
  • The README, manifests, CONTRIBUTING.md and a new SECURITY.md link to the OpenApps site and docs. Actions are pinned to commits.

Results

Round Before After
Claude, Free and Pro 26 of 36 36 of 36
Codex, Free and Pro 12 of 36 30 of 36
Signed out, Claude 13 of 21 20 of 21
Routing, 3 runs per query, optimize-listing 24 of 27 27 of 27
Routing, audit-discoverability 22 of 30 26 of 30
Routing, plugin-stack 16 of 21 21 of 21

Part of the gain comes from correcting checks that were too strict. The "after" numbers combine the full round with reruns of the journeys that the last fixes touched. The signed-out miss is a routing miss, on a request whose answer was correct. Its description fix passed on Free and Pro and wasn't rerun signed out. The six Codex misses are mostly single checks, and Chromium wouldn't start in Codex's sandbox for the widget journeys. SF2 on Free still leaves out a keep verdict for unchanged fields.

scripts/test-installs.sh, check-skills and check-contract pass.

…ck to OpenApps

Verification rounds under Claude and Codex, signed out and on Free and Pro,
drove these changes:

- Plan limits, Pro's one-call reads and the budget line are in the references.
- A denied npx moves on to MCP or curl before the skill asks the user anything.
- Codex keeps .agents/ read-only, so the context file falls back to the root.
- A how-do-I question runs submit-plugin and audit-discoverability on the repo.
- Skill descriptions cover the requests that used to miss them.
- check-contract fails when a reference drifts from the live API.
- The README, manifests and SECURITY.md link to the OpenApps site and docs.
@ataniz
ataniz merged commit 9a8d083 into main Nov 5, 2026
1 check passed
@ataniz
ataniz deleted the improvements-from-verification branch November 6, 2026 12:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant